When attending movies, people are accustomed to viewing relatively bright images within darkened theatres. By comparison, business projectors often provide modest luminance images on screens in dimmed conference rooms, such that image contrast is effectively low. As projection technologies evolve, and their uses expand, the range of different viewing experiences will expand as well. In particular, pico-projectors, micro-projectors, and other portable projectors, may be used in a variety of circumstances where the image size, screen luminance, and ambient viewing conditions are mutable. In such instances, the output flux is rather limited, and the brightness (or luminance, in ft·L or cd/m2) can change dramatically with screen distance (image size), and spans the entire range of visual adaptation, from the photopic range, through the mesopic range, and into the scotopic range. These are the three major ranges of visual adaptation, reflecting changes in the overall brightness sensitivity of the human visual system, as currently understood.
It is also understood that the human visual system adapts to changes in the overall color of illumination to preserve color constancy, which refers to the fact that color stimuli tend to retain their color appearance under a change of illuminant. Thus, for example, a page of white paper is perceived as white, whether viewed under daylight or tungsten (blue deficient) illumination conditions. However, this chromatic adaptation, while significant, is approximate. As a result, color perception by the viewers can then change too, as both brightness adaptation and chromatic adaptation occur during the course of changing illumination conditions.
As a baseline, cinematic projection is specified to provide 16 ft·L (foot-Lamberts) or 55 cd/m2 (candles per m2) of peak luminance, which is at the low end of the photopic range. Photopic vision is commonly defined as the vision of the eye that occurs under well-lit conditions (luminance levels of ˜3.5 to 106 cd/m2). When projected, image content causes light modulation that can change luminance values so that they fall into the mesopic range. The mesopic visual range is generally accepted to occur when visual stimuli have luminances less than several cd/m2, but greater than several hundredths of a cd/m2 (for example, 0.01 cd/m2 to 3.5 cd/m2). As the typical movie reduces the average screen luminance by ˜10×, to ˜1.6 ft·L, the apparent screen brightness is typically at the high end of the mesopic range. However, luminance levels can drop further, into the scotopic range, with dark image content and/or underlit projectors. Luminances below the mesopic range are said to fall into the scotopic range of adaptation. Although cinematic projection rarely stays in the scotopic range long enough for the viewers eyes to become night vision adapted, vision adaptation among audience members for mesopic viewing is common. However, in the case of cinema, the cinematographer subjectively corrects for this eye adaptation by viewing the content in a screening room (a dark environment, but smaller than a theatre), and then makes decisions on lighting and other production factors, to get the desired look, including color appearance. In the traditional film system, these decisions are carried forward, in illuminant adjustments within color printers when release prints are made at film laboratories, such as Technicolor®.
Similarly, the colorist in a telecine suite adjusts the illumination, or the electronic color settings (gain, LUT, etc.), to optimize the film to video transfer for television viewing. In that case, the goal is to provide a color viewing experience on a television, which holds close to the cinematographer's original intent for theatre viewing. However, the colorist works in an environment with a brighter ambient lighting than a screening room or theatre, that is generally equivalent to in-home lighting levels. The colorist also works with displays that have a brighter screen luminance than theatres (televisions are specified to provide 120 cd/m2 or 35 ft·L peak luminance), but that provide less image contrast and a diminished color gamut. Similar methods are used in optimizing direct digital “Hollywood” type content.
The fact that cinematographers and colorists make such efforts to color compensate for the differences in viewing conditions, including the changing visual color response to light level, is indicative that these changes are significant. Although the cinematographer and colorist recognize that the content is often viewed in sub-prime conditions, they have optimized the content for the standardized viewing conditions they wish were present.
Outside of the cinematic environment, or the standardized television environment, viewing conditions and perceptual differences vary dramatically. As one example, companies such as Microvision (Redmond, Wash.) are introducing low lumen pico-projectors, which can project in-focus images of different sizes, depending on the projector to screen distance. For example, a 10 lumen pico projector can provide a bright image (16 ft. L or 55 cd/m2) over a small image area (<1 ft2). If this image is modulated down by content, it would remain in the photopic zone for the first 20:1 modulation, and slip into a mesopic range below that. This case would be similar to cinema projection on a small scale. However, as such a projector is inherently portable, it can readily be used to project onto a large area, albeit to provide a dim image. For example, a 10 lumen projector illuminating a large area (10 ft2) would start projection with a peak luminance of ˜1 ft·L or ˜3.4 cd/m2, which corresponds to a common definition of the photopic/mesopic transition point. As mesopic vision has a ˜300-350:1 dynamic range, often defined from 0.011-3.4 cd/m2, image content modulation can easily extend deep into the mesopic visual range, or below it, into the scotopic range. In such cases, the ambient lighting conditions should be reduced if possible, to provide better viewing conditions, which can drop overall viewing conditions into the mesopic range or lower. In such instances, the viewers will experience significant brightness adaptation, and their color perception will also change. It would be useful to then modify the color projection properties provided by the projector, to compensate for the change in color perception with associated with brightness adaptation to low or varying luminance levels, and thus to provide a more consistent color viewing experience.
The prior art contains examples of altering images, whether hardcopy or electronically provided, for variable viewing conditions. As one example, U.S. Pat. No. 5,754,682 (Katoh) describes a picture processing apparatus in which output images can be produced such that a soft copy (electronic) image can coincide in appearance with a hard copy image, while taking into account both the ambient and electronic display brightness. In this case, no attempt is made to improve the images or to account for lack of colorfulness in both images, since the primary goal is to match the appearance of the hardcopy and softcopy images to the observer. Additionally, the luminance levels of interest are in excess of 100 cd/m2, and color appearance correction for mesopic viewing is not anticipated.
As another example, commonly assigned U.S. Pat. Nos. 6,411,306 and 6,529,212 (both by Miller et al.), describe an apparatus for automatically controlling the output luminance and image contrast provided by a display device, when taking into account eye adaptation. The apparatus has sensors to measure both the ambient illumination of the viewing environment and the output luminance of the display device. The resulting data is provided to a computer that produces a compensating signal that continually adjusts the luminance and contrast of the displayed image so as to prevent changes in the brightness and contrast of the image as perceived by the viewer under different ambient conditions. The computer also takes into account the viewing conditions, relative to potential adaptation in the eyes of the viewers. As visual sensitivity increases with eye adaptation, the apparatus can prevent changes in the perceived brightness and contrast of the images, by modifying screen brightness (luminance) and image contrast in a manner that compensates for the eye adaptation changes. However, these patents do not describe how to adjust the display or improve the image for very dim display luminances. In particular these patents do not provide color image correction for viewers experiencing mesopic viewing conditions, for whom perceived color hues are not constant and color saturation is especially weak.
Prior art U.S. Pat. No. 7,142,218 (Yoshida et al.), describes a display device equipped with sensors to determine the spectral composition of the ambient light with respect to chromaticity coordinates. The displayed images are altered using a process that converts an input chrominance signal to a different output chrominance signal, based on the characteristics of the external ambient illumination. Additionally, the target color chrominance signals on the display can be adjusted to provide images that are corrected for human chromatic adaptation characteristics. In particular, this method is primarily concerned with compensating displayed images for changes in how the human visual system adapts color vision in response to changes in the spectral content of the ambient illumination. However, Yoshida does not provide guidance for adjusting displayed image content appropriately to compensate for the changing response of human vision, when the absolute level of luminance decreases into the mesopic range.
As another example, prior art U.S. Pat. No. 7,499,163 (Lianza et al.) describes a system for implementing an appearance model correction for a display, which includes means for measuring an ambient illuminance associated with a display, means for calculating a display correction based in part on the measured ambient illuminance using a polynomial-based algorithm, and means for implementing the calculated correction on the display. The polynomial-based correction is an empirical fit to the more complex CIECAM02 color appearance model that has been certified by the CIE (Commission Internationale de L'Eclairage). Although the empirical fit provided by Lianza et al., extends to low phototopic luminance levels, it does not include luminance levels below 3 cd/m2 (see Table 1 of Lianza et al.). In addition, according to the CIE Activity Report for Division 1, Vision and Color of January 2008 (page 26, Extensions of CIECAM02), research work is just beginning to address the issue of extending the CIECAM02 model into the mesopic region of vision. Therefore the adjustment method of Lianza et al. cannot adequately address the situation of very low display luminances.
Finally, prior art U.S. Pat. No. 6,975,776 (Ferguson) provides a method for predicting variations in human perception under different luminance conditions, and then using perceptual difference calculations with respect to a reference visual model, to thereby determine corrective spatio-temporal filters that can be applied to video signals. While Ferguson provides compensating approaches for dark adaptation (night vision) relative to pupil size or photon noise, contrast, and correlation, image corrective methods for color attributes for viewers under low luminance or mesopic conditions are not addressed or anticipated.
Thus, as projectors and other displays become increasingly portable, and more likely to display images under low luminance conditions, visual perception of the displayed content will suffer. In particular, all aspects of color perception, related to luminance, saturation, and hue, are altered in the mesopic range. Therefore, it is desirable to enhance image display to viewers, relative to color perception, during the course of image display under mesopic viewing conditions.